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DETAILED ACTION 

1 . This office action is responsive to communication filed on February 28, 2008. 
Claims 1-12 are pending in the application and have been examined by the Examiner. 

Continued Examination Under 37 CFR 1.114 

2. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
February 28, 2008 has been entered. 

Response to Arguments 

3. Applicant's arguments filed February 28, 2008 have been fully considered but 
they are not persuasive. 

4. With respect to claims 1,2,4, 6, 8, 9, 1 1 and 1 2, Applicant argues that the 
Examiner mischaracterizes what Hashima teaches with respect to what Applicant 
teaches in the claims. Applicant then provides examples of how the invention of 
Hashima and the current invention differ, which examples pertain to the current 
invention repositioning an imaging device over time intervals of months or years using 
digital calculations. 
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5. In response to Applicant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies 
(i.e., the time interval of the repositioning, and the digital calculations) are not recited in 
the rejected claim(s). The Applicant is reminded that although the claims are 
interpreted in light of the specification, limitations from the specification are not read into 
the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). 
With regards to the mischaracterization, the Examiner respectfully disagrees. During 
examination, claims are to be given their "broadest reasonable interpretation" (see 
MPEP §2111). The Examiner upholds that one having ordinary skill in the art at the 
time of the invention would have been able to interpret the claims as currently written to 
be the invention taught by Hashima et al. 

6. Applicant argues that VanBree uses the term "reference image" to refer to the 
computational model extracted from an initial image of the target of interest. Hashima 
uses the term "reference image" to refer to an actual image to which later images are 
compared. 

7. Claims 1 and 4 read, "a reference image of a subject of interest". Therefore, the 
actual image taught by Hashima clearly reads on this limitation. Furthermore, regarding 
the statement that the current invention uses the term "reference image" to refer to the 
computational model extracted from an initial image of the target of interest, the claims 
are interpreted in light of the specification, limitations from the specification are not read 
into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 
1993). 
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8. Applicant argues that VanBree teaches, as shown in Fig. 3, a method for creating 
a reference image with no requirement for alignment of the camera (imaging device) to 
an arbitrary set of targets. 

9. The Examiner acknowledges that the methods of creating the reference image 
may differ between the Hashima reference and the current invention. However, these 
differences are not clearly illustrated in the present claims. 

1 0. Applicant argues that Hashima teaches of performing a complex calibration 
before the positional shift calculating unit is capable of communication position 
adjustments to reposition the image capture device. 

1 1 . The Examiner acknowledges that the method of Hashima may be more complex. 
However, this is a moot point, as Hashima still teaches of using a reference image in 
association with a computational device to reposition an image capture device as 
currently claimed by Applicant, and thus anticipates Applicant's invention. 

12. Applicant argues that VanBree teaches performing the repositioning using a 
single computation. 

1 3. In response to Applicant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies 
(i.e., that only a single computation is performed) are not recited in the rejected claim(s). 
Although the claims are interpreted in light of the specification, limitations from the 
specification are not read into the claims. See In re Van Geuns, 988 F.2d 1 181 , 26 
USPQ2d 1057 (Fed. Cir. 1993). Furthermore, the idea of a "single computation" is 
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broad and could range from a single equation to a complex program that is run a single 
time. 

14. Applicant argues that VanBree's invention can perform a comparison and 
communicate position adjustments to reposition the image capture device from a 
reference image and a single new image of the scene. 

1 5. The Examiner asserts that this embodiment is taught by Hashima, as coordinates 
are extracted from a single image to determine a shift from the target mark(column 7, 
lines 45-64). Furthermore, Hashima teaches that shifts between the reference image 
and the current image can be determined using a single image of the target mark 
because different points on the target mark correspond to different alignment 
references(column 1 6, lines 1 1 -37). See also column 1 8, line 42 through column 1 9, 
line 24. Hashima teaches of determining shifts based on, "an image of the target mark", 
as opposed to the multiple images as asserted by Applicant. 

16. With regards to claims 3, 7 and 1 0, Applicant argues that Verghese only applies 
to two dimensional models. 

1 7. The Examiner asserts that there is nothing in claims 3, 7 and 1 0 that limits said 
claims to three dimensions. Verghese simply teaches the benefits of a user interface, 
and of using more than one camera center as required by the claims. 

18. With respect to claim 5, Applicant argues that Palm merely teaches a simplified 
method that can be used to extract a model of a 3-dimensional object from multiple 
camera views of an object using only three points whose relative positions are known 
precisely, and that this is different from the current invention. 



Application/Control Number: 10/811 ,01 9 Page 6 

Art Unit: 2622 

19. The Examiner upholds that Palm teaches generating a three dimensional model 
of the subject of interest through selection of reference pointes in the subject of interest 
as currently required by claim 5. As Palm allegedly teaches using only three points, 
said three points can clearly be interpreted as "reference points of a subject of interest". 
As Hashima teaches imaging a target mark of many known points, the reference points 
taught by Palm can clearly correspond to the known points taught by Hashima. 

20. Therefore, the rejection is maintained by the Examiner. 

Claim Rejections - 35 USC § 102 

21 . The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

22. Claims 1, 2, 4, 6, 8, 9, 11 and 12 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Hashima et al.(US 5,521 ,843). 

23. The response to Applicant's arguments, as outlined above, is hereby 
incorporated into the rejection of claims 1 , 2, 4, 6, 8, 9, 1 1 and 12 by reference. 

Consider claim 1 , Hashima et al. teach: 

An imaging system (figure 1) to reposition an image capture device(camera, 20) 
in a position relative to a subject of interest according to six degrees of freedom(column 
7, lines 38-65) as preserved in association with a reference image("image produced 
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when the target mark 10 is in the target position", column 16, lines 9-16) of the subject 
of interest(see figure 1), comprising: 

an image capture device(20, figure 1); 

a position apparatus(robot, 30) on which the image capture device(20) is 
mounted(see figure 1), operable to orient the image capture device(20) relative to a 
subject of interest according to six degrees of freedom(column 7, lines 38-65); 

a reference image of the subject of interestfimage produced when the target 
mark 10 is in the target position", column 16, lines 9-16); 

a computational device(50, 60) coupled to the position apparatus(30, see figure 
1), such computational device(50, 60) capable of receiving images from the image 
capture device(20) and receiving the reference image, performing a comparison, and 
communicating adjustments to reposition the image capture device(20) along any of six 
degrees of freedom(A current image is compared with a reference image, a difference is 
calculated, this difference is sent to the robot controller(60), and the robot controller(60) 
controls the robot(30) to position the camera(20) such that the current image position is 
the same as the reference image position. See column 16, line 9 through column 19, 
line 26.). 

Consider claim 2, and as applied to claim 1 above, Hashima et al. further teach 
that the communication of position adjustments is via signals to the positional 
apparatus(30) from the computational device(50, 60, column 7, lines 45-65, column 18, 
line 47 through column 19, line 24). 
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Consider claim 4, Hashima et al. teach: 

A method for repositioning an image capture device(20) relative to a subject of 
interests ) according to six degrees of freedom (column 15, line 58 through column 19, 
line 26, figure 29) comprising the steps of: 

a) initializing an imaging system, wherein initializing includes the steps of: 
a. 1) obtaining a reference image of the subject of interestfimage produced 

when the target mark 10 is in the target position", column 16, lines 9-16) wherein said 
reference image includes multiple reference points in 3-dimensional space(See column 
7, lines 45-48, column 7, line 66 through column 8, line 17, column 15, line 58 through 
column 16, line 8, figures 2, 3, and 4. A reference image is obtained of a three- 
dimensional target mark(10).); 

a. 2) repositioning an image capture device relative to the subject of interest, 
where such repositioning uses six degrees of freedom(column 16, lines 9-57); 

b) imaging the subject of interest(column 16, lines 9-16); 

c) computing the difference between the reference image of the subject of 
interest and the image capture device image(column 16, line 9 through column 18, line 
30, note especially column 16, lines 9-16); 

d) refining the position of the image capture device(20) so that the image capture 
device(20) is in the same position relative to the subject of interest as that position from 
which the reference image was obtained, where such refining the position of the image 
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capture device occurs along six degrees of freedom(column 16, lines 9-16, column 18, 
line 31 through column 19, line 26). 

Consider claim 6, and as applied to claim 4 above, Hashima et al. further teach 
that the reference image is obtained after fixed reference points have been selected in 
the subject of interest(See figures 2, 3, and 4, column 7, line 66 through column 8, line 
17. A target mark(10) having fixed reference points is placed in the image and captured 
with the reference image.). 

Consider claim 8, and as applied to claim 4 above, Hashima et al. further teach 
that time has elapsed between the initialization process and the repositioning of the 
image capture device(Column 15, line 62 through column 16, line 16. A reference 
image is obtained with the target mark(10) in the target position, and later compared to 
a recent image to reposition the camera.). 

Consider claim 9, and as applied to claim 4 above, Hashima et al. further teach 
that the computation of position is communicated to an automatic position correction 
apparatus(robot, 30, figure 1 , column 7, lines 38-56, column 18, line 36 through column 
19, line 24). 



Consider claim 1 1 , Hashima et al. teach: 
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An apparatus(30, figure 1) for positioning an imaging device(20) and adapted for 
operably coupling to an image capture device(20, see figure 1) and where such 
apparatus(30) is capable of positioning said image capture device(20) along six degrees 
of freedom(column 7, lines 38-65), such that the positioning of the image capture 
device(20) is controllable and said apparatus(30) is operable to orient the image capture 
device(20) relative to a subject of interest using six degrees of freedom to orient the 
image capture device(column 7, lines 38-65). 

Consider claim 12, and as applied to claim 11 above, Hashima et al. further 
teach that the positioning of the image capture device is automated(The positioning is 
done by a mechanical robot(30), column 7, lines 38-65, figure 1 .) 

Claim Rejections - 35 USC § 103 

24. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

25. Claims 3, 7 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Hashima et al. in view of Verghese(US 7,038,709). 

26. The response to Applicant's arguments, as outlined above, is hereby 
incorporated into the rejection of claims 3, 7 and 10 by reference. 
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Consider claim 3, and as applied to claim 1 above, Hashima et al. teach of an 
imaging system to reposition an image capture device in six degrees of freedom(see 
claim 1 rationale). However, Hashima et al. do not explicitly teach of a user interface. 

Verghese is similar to Hashima et al. in that Verghese teaches of an imaging 
system(figures 1-3) to reposition an image capture device(Camera, 16) in a position 
relative to a subject of interest as that of a reference image of the subject of interest, 
comprising an image capture device(camera, 16), a position apparatus(figure 2) on 
which the image capture device(16) is mounted(see figure 3a), operable to orient the 
image capture device relative to a subject of interest(See column 5, lines 31-45. The 
position apparatus orients the image capture device in order to track the motion of the 
subject of interest.), a reference image of the subject of interest(See figure 12, step 508, 
column 18, lines 8-25. A reference image is obtained to determine current camera 
orientation.), a computational device(44, figure 1) coupled to the position 
apparatus(figure 1 ), such computational device(44) capable of receiving images from 
the image capture device(16) and of receiving the reference image(column 5, lines 56- 
67), performing a comparison(The image processing component(44) receives an image, 
determines the location of a certain color using a color tracking algorithm, centers that 
location on the camera field of view, compares subsequent frames to determine if the 
position of the predetermined color has moved from the center, and repositions the 
imaging device so that the predetermined color is re-centered. See column 5, line 56 
through column 7, line 12, figure 12, column 17, line 8 through column 19, line 6.), and 
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communicating position adjustments to reposition the image capture device(column 6, 
lines 37-55). 

However, in addition to the teachings of Hashima et al., Verghese teaches that 
the communication of position adjustments is by means of positional adjustment data 
conveyed by means of a user interface(column 5, lines 47-55, column 7, lines 24-35). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to include a user interface to communicate position 
adjustments as taught by Verghese in the imaging system to reposition an image 
capture device as taught by Hashima et al. for the benefit creating a more versatile 
device by allowing the user to control the size and quality of a displayed image, and the 
ability to override the image tracking system in favor of user positioning when 
desired(Verghese, column 5, lines 47-55). 

Consider claim 7, and as applied to claim 4 above, Hashima et al. teach of an 
imaging system to reposition an image capture device in six degrees of freedom, and of 
extracting reference points from an image(see claim 4 rationale). However, Hashima et 
al. do not explicitly teach that more than one image of the subject of interest 
representing more than one camera center are extracted. 

Verghese is similar to Hashima et al. in that Verghese teaches of an imaging 
system(figures 1-3) to reposition an image capture device(Camera, 16) in a position 
relative to a subject of interest as that of a reference image of the subject of interest, 
comprising an image capture device(camera, 16), a position apparatus(figure 2) on 
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which the image capture device(16) is mounted(see figure 3a), operable to orient the 
image capture device relative to a subject of interest(See column 5, lines 31-45. The 
position apparatus orients the image capture device in order to track the motion of the 
subject of interest.), a reference image of the subject of interest(See figure 12, step 508, 
column 18, lines 8-25. A reference image is obtained to determine current camera 
orientation.), a computational device(44, figure 1) coupled to the position 
apparatus(figure 1), such computational device(44) capable of receiving images from 
the image capture device(16) and of receiving the reference image(column 5, lines 56- 
67), performing a comparison(The image processing component(44) receives an image, 
determines the location of a certain color using a color tracking algorithm, centers that 
location on the camera field of view, compares subsequent frames to determine if the 
position of the predetermined color has moved from the center, and repositions the 
imaging device so that the predetermined color is re-centered. See column 5, line 56 
through column 7, line 12, figure 12, column 17, line 8 through column 19, line 6.), and 
communicating position adjustments to reposition the image capture device(column 6, 
lines 37-55). 

However, in addition to the teachings of Hashima et al., Verghese teaches that 
the step of initializing includes extracting reference points from more than one image of 
the subject of interest representing more than one camera center(Many images are 
obtained(column 5, lines 56-67), which images contain the same reference points, and 
these images contain more than one camera center as the camera is repositioned by 
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the positioning device to re-center the reference points in the varying images, column 6, 
line 1 through column 7, line 12). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to extract reference points from more than one image as 
taught by Verghese, when extracting reference points as taught by Hashima et al. for 
the benefit of being able to effectively and efficiently track the movement of a 
subject(Verghese, column 2, lines 7-12). 

Consider claim 10, and as applied to claim 4 above, Hashima et al. teach of an 
imaging system to reposition an image capture device in six degrees of freedom(see 
claim 4 rationale). However, Hashima et al. do not explicitly teach of a user interface. 

Verghese is similar to Hashima et al. in that Verghese teaches of an imaging 
system(figures 1-3) to reposition an image capture device(Camera, 16) in a position 
relative to a subject of interest as that of a reference image of the subject of interest, 
comprising an image capture device(camera, 16), a position apparatus(figure 2) on 
which the image capture device(16) is mounted(see figure 3a), operable to orient the 
image capture device relative to a subject of interest(See column 5, lines 31-45. The 
position apparatus orients the image capture device in order to track the motion of the 
subject of interest.), a reference image of the subject of interest(See figure 12, step 508, 
column 18, lines 8-25. A reference image is obtained to determine current camera 
orientation.), a computational device(44, figure 1) coupled to the position 
apparatus(figure 1), such computational device(44) capable of receiving images from 
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the image capture device(16) and of receiving the reference image(column 5, lines 56- 
67), performing a comparison(The image processing component(44) receives an image, 
determines the location of a certain color using a color tracking algorithm, centers that 
location on the camera field of view, compares subsequent frames to determine if the 
position of the predetermined color has moved from the center, and repositions the 
imaging device so that the predetermined color is re-centered. See column 5, line 56 
through column 7, line 12, figure 12, column 17, line 8 through column 19, line 6.), and 
communicating position adjustments to reposition the image capture device(column 6, 
lines 37-55). 

However, in addition to the teachings of Hashima et al., Verghese teaches that 
the computation of position is communicated to the user through an interface(column 5, 
lines 47-55, column 7, lines 24-35). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to include a user interface to communicate the computation 
of position as taught by Verghese in the imaging system to reposition an image capture 
device as taught by Hashima et al. for the benefit creating a more versatile device by 
allowing the user to control the size and quality of a displayed image, and the ability to 
override the image tracking system in favor of user positioning when desired(Verghese, 
column 5, lines 47-55). 

27. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hashima 
et al. in view of Palm(US 5,699,444). 
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28. The response to Applicant's arguments, as outlined above, is hereby 
incorporated into the rejection of claim 5 by reference. 

Consider claim 5, and as applied to claim 4 above, Hashima et al. teach of a 
method for repositioning an image capture device(20) relative to a subject of interest(1) 
according to six degrees of freedom(column 15, line 58 through column 19, line 26, 
figure 29, claim 4 rationale). Hashima also teach of calculating the position of the 6 
degrees of freedom based on a three-dimensional target(see claim 4 rationale). 

However, Hashima et al. do not explicitly teach the step of generating a three 
dimensional model of the subject of interest through selection of reference points in the 
subject of interest. 

However, as indicated by Palm, the repositioning of a camera using a three- 
dimensional model is well known in the art. Palm is similar to Hashima et al. in that 
Palm is also concerned with repositioning a camera to re-center a subject of 
interest(column 1 , lines 6-10, column 7, lines 26-39). 

In addition to the teachings of Hashima et al., Palm teaches of using three- 
dimensional coordinates of reference points, and thereby using three-dimensional 
models to reposition and re-center a subject in relation to a camera. See figures 8 and 
9, column 12, lines 21-48, column 15, lines 18-53. 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use three-dimensional models to reposition an imaging 
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system as taught by Palm in place of the target mark system taught by Hashima et al. 
for the benefit of providing simple, yet accurate procedures that can be applied 
successfully by non-technical personnel(Palm, column 4, lines 39-52). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ALBERT H. CUTLER whose telephone number is 
(571)270-1460. The examiner can normally be reached on Mon-Thu (9:00-5:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ngoc-Yen Vu can be reached on (571)-272-7320. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



AC 



/Ngoc-Yen T. VU/ 
Supervisory Patent Examiner, Art Unit 2622 
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